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Abstract 


covariance  from  a finite  state  continuous 

Markov  process.  Specifically  we  ask  the  following. 

Question : Given  a stationary  covariance  <ti(t,T) 
under  what  circumstances  does  there  exist  a finite 
state  continuous  time  Harkov  process  x(’)  caking 
on  values  in  a finite  set  X and  a function  f:X->TR 
(the  real  numbers)  such  that 

(a)  The  transition  probabilities  are  given 
by  a time  invariant  infinitesimal  gener- 
ator whose  null  space  is  one  dimensional. 

(b)  lim  E f[x(t)]  • 0 for  all  x(0) 

^-♦CO 

(c)  11m  E f [x(t)  ]f  [x(t+T)  ] • (!)(t)  for  all  x(0) 

t-Ko 

The  reason  for  assuming  that  A has  a one  dim- 
ensional kernel  is  so  that  regardless  of  the  initial 
state  of  the  x-process,  the  steady  state  density  is 
the  same . 

In  this  paper  we  derive  necessary  and  sufficient 
conditions  for  i)>  to  be  realizable  in  this  way. 
Roughly  speaking,  what  we  find  is  that  any  non- 
periodic stationary  covariance  can  be  approximated 
by  one  which  is  realizable  in  this  way.  Moreover 
those  covariances  with  rational  power  spectra  which 
can  be  realized  exactly  are  characterized. 

Since  one  knows  [4]  that  any  finite  state 
continuous  time  Markov  process  Is  equivalent  to  one 
which  can  be,  expressed  as 


Injhls  paper  we  -otnd^  the  stationary  covariance 

generation  problem,  l.e.  the  problem  of  passing 

from  a stationary  covariance  function  to  a 
dynamical  system  which  generates  a process  having 
Che  given  covariance,  in  Che  case  where  the  dynam- 
ical system  is  a finite  state,  continuous  time, 
Markov  process.  ,iWe— ftn^ thet  strictly  positive 
definite  stationary  covariances  can  be  approximated 
to  any  degree  of  accuracy  in  this  way.  However 
the  number  of  states  required  may  approach  Infinity 
as  the  covariance  approaches  the  boundary  of  the 
set  of  positive  definite  functions. 


1.  Introduction 


The  general  area  of  stochastic  realization 
theory  consists  of  results  on  how  to  pass  from  a 
certain  complete,  or  partial,  description  of  a 
stochastic  process  to  a differential  equation  (or 
difference  equation)  model  for  it  with  the 
stochastic  process  which  drives  the  differential 
equation  being  simpler,  or  more  "fundamental", 
than  the  original  process.  In  the  case  where  the 
given  partial  description  is  a stationary  covar- 
iance ij)  and  of  a constant  mean  m it  has  been  known 
for  some  time  under  what  circumstances  there  exists 
a realization  in  the  form  of  a finite  dimensional 
linear  Ito  equation 

dx(t)  - Ax(t)dt+bdw(t)+rdt;  y(t)”cx(t) 

Ey(t)  ■■  m 

(Kt,!)  - Hc-tj  = E(y(t)-m)(y(T)-m) 


with  N^(t)  a standard  Poisson  counting  process 
with  rate  X^,  this  work  is  a natural  complement 
to  the  Gauss-Markov  covariance  generation  problem 


with  constant  coefficients.  This  result,  which  has 
its  origin  in  the  Bode-Shannon  pre-whitening  filter 
states  that  is  realizable  if  it  is  an  even, 
L2(“‘”»”)  function  whose  Fourier  transform  (power 
spectrum)  is  rational  and  nonnegative  on  the  real 
axis.  One  then  finds  the  system  parameters  A,b,c 
and  r by  a factorization  of  the  power  spectrum. 
Since  the  power  spectrum  is  automatically  nonnega- 
tive this  result  is  quite  satisfactory;  the  only 
price  being  paid  for  finite  dimensionality  of  x is 
the  rationality  of  the  power  spectrum. 

In  this  paper  we  consider  covariance  gener- 
ation but  with  a view  toward  generating  a given 


2.  The  Model 

We  find  it  convenient  to  associate  with  each 
state  of  an  n state  Markov  process  a particular 
point  in  This  lets  us  visualize  the  process 

as  Jumping  between  points  in  a vector  space  and 
allows  us  to  use  certain  very  familiar  formulas 
from  linear  system  theory.  Let  ej^  be  the  ith 
standard  basic  element  in  7^  (prime  denotes 
transpose) 

ej  - [0,0,... ,0,1,0,. ..,0]’ 

t_lth  coordinate 

and  let  x be  a process  which  takes  on  the  values  in 
the  set  {e^,e2<...,e  }.  If  p, (t)  is  the  probability 
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that  x(t)“ej^  then  our  assumptions  Imply  that  there 
exists  a constant  matrix  A such  chat 


P]^(t)' 

®11  ®12  •' 

’ • “in 

■pj^(t)' 

®21  ®22  ■ 

“2n 

P2(t) 

p;(t). 

• ®nl  ®n2  • ' 

. . a 

nn 

,o’(t). 

n 

0 0 . ..  p , 

n 


with  (p, ,P2>.->p  ) being  the  probability  vector 
In  the  Remel  of"A. 


Notice  that  because  of  the  way  we  have  embedded 
the  states  In  we  have 

p(C)  = Ex(t) 

According  to  the  well  known  theory  of  such 
processes,  see,  e.g.  Karlin  [1],  the  entries  of  A 
satisfy  the  condition 

(i)  = j > 0 If  1 ,i  j 


(ii) 


Such  matrices  are  called  here  Infinitesimally 
stochastic.  Of  course  Peron-Frobenlus  theory 
Implies  that  A has  a nontrivial  null  space.  If  we 
ask  that  the  null  space  be  one  dimensional  then  we 
are  assured  that  there  Is  a unique  steady  state 
probability  distribution.  For  these  reasons  we 
will  assume 

(ill)  the  kernel  of  A is  one  dimensional. 

Such  processes  are  called  irreducible. 

As  an  immediate  consequence  of  the  definit- 
ions we  see  that 


3.  Realization  with  A ClrculanC 

The  determination  of  what  functions  can  be 
expressed  In  the  form  required  by  equation  (S) 

Is  made  especially  hard  by  the  requirement  that  A 
be  Infinitesimally  stochastic.  To  get  around 
the  awkwardness  of  this  constraint  we  focus 
attention  on  a special  class  of  matrices.  By  a 
circulant  matrix  we  understand  a square  matrix  of 
the  form 


1 0 if  i - j 
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Associated  with  each  such  M there  is  a polynomial 
m(z)-m^+mj^z+. . .-Hn^  The  eigenvalues  of  a 

circulant  matrix  are  simply  the  values 


ike. 

m(e  ) ; 


e 

n 


k-0,1 n-1 


Thus  M meets  condition  (1),  (11),  (ill)  if  and 
only  if 

the  coefficients  of  m(z)  are  all 
nonnegative  except  for  the  constant  term. 

m(l)  - 0 


(!') 


(11') 

dll') 


Ex(t)  = e*''x(0) 


(E) 


m(z)  does  not  vanish  for  z an  nth  root 
of  unity  unequal  to  one. 


Because  the  1th  and  jth  components  of  x are  never 
simultaneously  nonzero  and  because  the  components 
take  on  only  the  values  zero  and  one 


■p^(t)  0 

0 ■ 

E(t)  - Ex(t)x'(t)- 

0 p^d) 

0 

0 0 

Moreover  an  elementary  application  of  Bayes'  rule 
yields 

Ex(t)x'(T)  - E(t)e*'^'^"'^^;  T > t (V) 

The  equations  (E)  and  (V)  together  with  one 
more  remark  will  yield  a restatement  of  our 
question  In  linear  algebraic  form.  The  additional 
remark  is  this.  Any  map  of  the  state  space  of  the 
Markov  process  Into  TK  is  of  the  form  c'x  for  some 
c in  "J?".  Simply  let  the  1th  component  of  c be 
f(ej^)  where  f : X 77t.  Bringing  these  remarks 
together  we  have  the  following  theorem. 

Theorem  1:  A stationary  covariance  (()(t-T)-<))(t,T) 
Is  realizable  by  a zero  mean  process  which  Is  a 
function  of  a finite  state,  continuous  time 
Irreducible  Markov  process  if  and  only  if 

«(t)  - c'Ee*^c;  t > 0 (S) 

for  some  pair  (A,c)  where  A Is  an  n by  n matrix 
satisfying  (1),  (11),  (ill),  c is  an  n vector  such 
that  c'x”0  for  all  x In  the  kernel  of  A and 


Under  these  assumptions  one  sees  easily  that  the 
solution  of 

p = l(p 

for  p(0)  a probability  vector,  tends  to 

rii 


Thus  using  such  an  M we  see  that  the  matrix  E in 
equation  (S)  becomes  i I and  we  have 

♦(t)  ’ ^ c'e”^:  t > 0 

The  condition  that  the  mean  should  vanish, c'p  *0, 
is  also  easily  interpretable.  In  fact  if 

c'  . (c^,c^,...c^_^) 

we  introduce 

c(z)  - c +c,z+...+c  , z"  ^ 

o 1 n-i 

In  this  notation  c'p  ”0  !>ecomes 
o 

(b')  c(l)  - 0 

We  can  also  express  (ti  succinctly  in  terms  of 
m(z)  and  c(z)  (see  [3]). 

J(t)  - \ c(z)c(z-l)e”^*>S  t > 0 

{z:z"-l} 
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cos  utdlJ 


The  following  lemma  gives  a somewhat  more  satis- 
factory form  of  this. 


Lemma  1:  The  set  of  finite  state  continuous  time 
realizable  covariances  Includes  those  covariances 
expressible  as 


<>(t) 


n-1 

I 

k-1 


m(e 


2Tilk/n)t 


t » 0 


with  m satisfying  (1').  (11')  and  (111')  and  the 
real  and  nonnegative.  In  particular 

i()(t)  " re  '^^cos  uit;  t k 0 


Is  so  realizable  If  r and  O are  real  and  positive 
and  (ii  Is  real. 

Proof:  Of  course  c(z)c(z  ^)  Is,  for  z on  the 
unit  circle,  real  and  nonnegatlve.  Since  c(l)*0 
we  must  have  c(z)c(z”)')  vanishing  at  z“l  but  other- 
wise we  may  pick  the  coefficients  so  that  c(z)  has 
arbitrary  complex  values  at  the  nth  roots  of  unity 
consistent  with  c(z)  * c(D . Adding  up  the  con- 
tribution from  P • we  get 

ii,  (t)  • 2|c(p)|^e^  ”^*^^^008  Iffl  m(p)t 

But  since  c(p)  Is  arbitrary  we  see  that  |c(p)| 
can  be  any  real  nonnegatlve  number.  The  general 
form  given  In  the  lemma  then  follows. 

To  show  that  the  specific  tp  given  In  the 
lemma  Is  expressible  In  this  way  we  make  a 
particular  choice  of  n and  m(z).  Let 
2 

m(z)  • a(l-ciz-(l-a)2  );  a,o,  > 0 

and  let  a > 0 and  W be  given.  At  z-e^^^^"  « 
cos (2Tt/n) •Ksln(2n/n)  the  ratio  of  the  real  to  the 
Imaginary  parts  of  m Is 

l-aco8(2TT/n)-(l-CT)cos(4Tr/n) 

^ ’ -asln(2'iT/n)-(l-a)sln(4iT/n) 


Inspection  of  this  equation  shows  that  for  any 
negative  y we  can  choose  an  Integer  n large  enough 
so  as  to  have  a solution  for  a.  (As  y approaches 
zero  n(Y)  goes  to  Infinity.)  Thus  we  can,  with 
this  choice  of  m(z)  adjust  the  magnitude  and 
argument  of  m(exp  27Tl/n)  as  needed  to  get  the 
function  i|>(  ) of  the  lemma.  Of  course  we  pick  c 
In  such  a way  as  to  vanish  on  all  nth  roots  of 
unity  except  the  two  which  enter  In  this  discussion. 


4.  The  Approximation  Lemma 

Lemma  1 makes  It  clear  how  to  realize  non- 
negatlve linear  combinations  of  the  basic  terms 
labeled  there  tl>;  take  the  direct  svmi  of  realiza- 
tions of  the  type  constructed  In  Its  proof.  How- 
ever these  realizations 

(a)  will  not.  In  general^  satisfy  the 
Irreduclblllty  condition,  and 

(b)  give  no  suggestion  as  to  how  to  realize 

covariances  such  as  <>(t)”10e  ^^^-e  which  are 

differences  of  positive  definite  functions  but 
still  positive  definite. 

In  this  section  we  establish  the  results 
necessary  to  get  around  these  difficulties. 

If  it(t)  Isa  continuous,  even,  positive  def- 
inite function  then  according  to  the  well  known  re- 
presentation theorem  of  Bochner  it  can  be  expressed 
as 


■fd)  - 

J [0,®) 

for  some  nonnegatlve  measure  p.  Of  course  If  u Is 
absolutely  continuous  with  respect  to  Lebesgue 
measure  then  we  can  write 

f” 

(Kt)  “ cos  (ijt'l'(<i))du)  ; ti(ijj)  k 0 

Jo 

displaying  the  power  spectrum  explicitly. 

However,  If  we  assume  that  ^ Is  not  only 
positive  definite  but  In  addition  It  Is  strictly 
positive  definite  In  the  sense  that 

()>g(t)  - (f(t)e^^l''l 

Is  for,  some  e > 0,  also  square  Integrable  and 
positive  definite  Chen  we  can  express  p as 


(Kt) 


0 


^0  t I 

e 'cos  i»)t4>(iu)du) 


with  4’  analytic.  (This  follows  from  Payley- 
Wlener  theory;  the  Fourier  transform  of 

4i(t)e^^*^  analytic  In  a strip  of  width  2e  centered 
on  the  u>-axls  and  4’  Is  Its  Fourier  transform.)  Let 
be  any  finite  set  of  real  numbers.  We 
can  approximate  simultaneously  the  Integrals  if(lj^) 
by  Rlemann  sums  and  thus  obtain 
m 

(Kt)  " I e cos  u t(ti(u).)s,+e(t) 

1-1 

with  |c(t,)  I less  than  any  preassigned  positive 
number.  Both  ()>  and  the  approximation  go  to  zero 
as  ]tl  -<■  ®;  In  view  of  the  continuity  of  i)i 
we  see  that  It  can  be  uniformly  approxi- 
mated by  a linear  combination  of  i(>-llke  terms 
with  positive  coefficients.  The  following  theorem 
summarizes . 


Theorem  2:  Any  continuous,  strictly  positive  defi- 
nite function  Is  the  uniform  limit  of  a sequence 
()>„  of  the  form 
m(n) 

♦^(t)  • I cos  ui^d):  ^ ° 


We  now  address  the  second  problem  mentioned  at 
Che  start  of  this  section. 

Lemma  2 ; Stationary  covariances  of  the  form 
appearing  In  theorem  2 can  be  realized  by  a pair 
(A,c)  satisfying  conditions  (1),  (11),  and  (111). 

Proof:  Let  (A,  ,c.  ) be  a realization  of 
— £t/2 

Oj^e  cos  uj^t  of  the  form  given  In  the  proof  of 
lemna  1;  l.e.  of  the  clrculant  form.  (Note  we 
have  e/2  as  the  decay  factor.)  Then  for 


■*1 

0 

. ..  O' 

■<=i‘ 

0 

*2 

...  0 

; c ■ 

^2 

.6 

0 

. ..  A 

n 

c 

n J 

It  follows  Chat 

I At  r -ct/2 
c e c - I OLe  cos  ut 

k-1  “ 

Now  subtract  from  A the  Infinitesimally  stochastic 
matrix 
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F - ^ I - ^ 
2 2n 


ell 


where  n Is  the  sum  of  the  dimensions  of  the  A . 
Clearly  A-F  Is  Infinitesimally  stochastic  and 
Irreducible.  Notice  that 


c*e(A-eI/2)t, 


-et 


(This  time  the  decay  factor  agrees  with  theorem  2.) 
Finally,  becauje  of  the  null  spaces  of  each  of  the 
Aj^  we  see  that 


Together  these  Imply  c'e(*"^)^c“c'e^*  el/2)t 
(A-F,c)  meets  all  requirements. 


5.  On  the  Number  of  States 

In  response  to  a question  raised  by 
Kolmogorov,  about  30  years  ago  Dlmltrlev  and 
Dynkln  established  that  an  n by  n stochastic  matrix 
cannot  have  any  eigenvalues  In  the  part  of  the  unit 
circle  bounded  by  a chord  passing  through  the  point 
(1,0)  In  the  complex  plane  and  making  angles  of 
(2ir/p)+Tr/2  with  the  real  axis  (see  figure  lb) 
provided  p ) n Is  an  Integer  (see  [2]  for  refer- 
ences). Since  Che  exponential  of  an  Infinitesimal 


Dlmltrlev-Dynkln  result. 


stochastic  matrix  with  eigenvalues  ^n 

Is  a stochastic  matrix  with  eigenvalues 
exp  X ,exp  X2,...,exp  X this  Imposes  certain 
constraints  on  the  eigenvalues  of  an  Infinitesimally 
stochastic  matrix.  The  space  of  Infinitesimally 
stochastic  matrices  Is  a cone  and  so  the  excluded 
subset  of  the  complex  plane  must  be  bound  by 
straight  lines  passing  through  zero  (see  figure  la). 
Some  arithmetic  establishes  that  a+lu  cannot  be 
an  eigenvalue  of  an  n by  n Infinitesimally 
stochastic  matrix  unless 


a . 2n  . II 

— — Hn  cos 

10  IT  2n 


Theorem  3:  The  power  spectrum  associated  with  the 
steady  state  covariance  matrix  of  n-state  contin- 
uous time  Markov  process  Is  rational  and  has  no 
poles  In  the  region  of  the  complex  plane  for  which 
Inequality  (A)  Is  violated. 
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